Mathematical properties of neuronal TD-rules and differential Hebbian learning: a comparison

A confusingly wide variety of temporally asymmetric learning rules exists related to reinforcement learning and/or to spike-timing dependent plasticity, many of which look exceedingly similar, while displaying strongly different behavior. These rules often find their use in control tasks, for example in robotics and for this rigorous convergence and numerical stability is required. The goal of this article is to review these rules and compare them to provide a better overview over their different properties. Two main classes will be discussed: temporal difference (TD) rules and correlation based (differential hebbian) rules and some transition cases. In general we will focus on neuronal implementations with changeable synaptic weights and a time-continuous representation of activity. In a machine learning (non-neuronal) context, for TD-learning a solid mathematical theory has existed since several years. This can partly be transfered to a neuronal framework, too. On the other hand, only now a more complete theory has also emerged for differential Hebb rules. In general rules differ by their convergence conditions and their numerical stability, which can lead to very undesirable behavior, when wanting to apply them. For TD, convergence can be enforced with a certain output condition assuring that the δ-error drops on average to zero (output control). Correlation based rules, on the other hand, converge when one input drops to zero (input control). Temporally asymmetric learning rules treat situations where incoming stimuli follow each other in time. Thus, it is necessary to remember the first stimulus to be able to relate it to the later occurring second one. To this end different types of so-called eligibility traces are being used by these two different types of rules. This aspect leads again to different properties of TD and differential Hebbian learning as discussed here. Thus, this paper, while also presenting several novel mathematical results, is mainly meant to provide a road map through the different neuronally emulated temporal asymmetrical learning rules and their behavior to provide some guidance for possible applications

Exploring matrix factorization techniques for significant genes identification of Alzheimer’s disease microarray gene expression data

<p>Abstract</p> <p>Background</p> <p>The wide use of high-throughput DNA microarray technology provide an increasingly detailed view of human transcriptome from hundreds to thousands of genes. Although biomedical researchers typically design microarray experiments to explore specific biological contexts, the relationships between genes are hard to identified because they are complex and noisy high-dimensional data and are often hindered by low statistical power. The main challenge now is to extract valuable biological information from the colossal amount of data to gain insight into biological processes and the mechanisms of human disease. To overcome the challenge requires mathematical and computational methods that are versatile enough to capture the underlying biological features and simple enough to be applied efficiently to large datasets.</p> <p>Methods</p> <p>Unsupervised machine learning approaches provide new and efficient analysis of gene expression profiles. In our study, two unsupervised knowledge-based matrix factorization methods, independent component analysis (ICA) and nonnegative matrix factorization (NMF) are integrated to identify significant genes and related pathways in microarray gene expression dataset of Alzheimer’s disease. The advantage of these two approaches is they can be performed as a biclustering method by which genes and conditions can be clustered simultaneously. Furthermore, they can group genes into different categories for identifying related diagnostic pathways and regulatory networks. The difference between these two method lies in ICA assume statistical independence of the expression modes, while NMF need positivity constrains to generate localized gene expression profiles.</p> <p>Results</p> <p>In our work, we performed FastICA and non-smooth NMF methods on DNA microarray gene expression data of Alzheimer’s disease respectively. The simulation results shows that both of the methods can clearly classify severe AD samples from control samples, and the biological analysis of the identified significant genes and their related pathways demonstrated that these genes play a prominent role in AD and relate the activation patterns to AD phenotypes. It is validated that the combination of these two methods is efficient.</p> <p>Conclusions</p> <p>Unsupervised matrix factorization methods provide efficient tools to analyze high-throughput microarray dataset. According to the facts that different unsupervised approaches explore correlations in the high-dimensional data space and identify relevant subspace base on different hypotheses, integrating these methods to explore the underlying biological information from microarray dataset is an efficient approach. By combining the significant genes identified by both ICA and NMF, the biological analysis shows great efficient for elucidating the molecular taxonomy of Alzheimer’s disease and enable better experimental design to further identify potential pathways and therapeutic targets of AD.</p

An Imperfect Dopaminergic Error Signal Can Drive Temporal-Difference Learning

An open problem in the field of computational neuroscience is how to link synaptic plasticity to system-level learning. A promising framework in this context is temporal-difference (TD) learning. Experimental evidence that supports the hypothesis that the mammalian brain performs temporal-difference learning includes the resemblance of the phasic activity of the midbrain dopaminergic neurons to the TD error and the discovery that cortico-striatal synaptic plasticity is modulated by dopamine. However, as the phasic dopaminergic signal does not reproduce all the properties of the theoretical TD error, it is unclear whether it is capable of driving behavior adaptation in complex tasks. Here, we present a spiking temporal-difference learning model based on the actor-critic architecture. The model dynamically generates a dopaminergic signal with realistic firing rates and exploits this signal to modulate the plasticity of synapses as a third factor. The predictions of our proposed plasticity dynamics are in good agreement with experimental results with respect to dopamine, pre- and post-synaptic activity. An analytical mapping from the parameters of our proposed plasticity dynamics to those of the classical discrete-time TD algorithm reveals that the biological constraints of the dopaminergic signal entail a modified TD algorithm with self-adapting learning parameters and an adapting offset. We show that the neuronal network is able to learn a task with sparse positive rewards as fast as the corresponding classical discrete-time TD algorithm. However, the performance of the neuronal network is impaired with respect to the traditional algorithm on a task with both positive and negative rewards and breaks down entirely on a task with purely negative rewards. Our model demonstrates that the asymmetry of a realistic dopaminergic signal enables TD learning when learning is driven by positive rewards but not when driven by negative rewards

Temporal-Difference Reinforcement Learning with Distributed Representations

Temporal-difference (TD) algorithms have been proposed as models of reinforcement learning (RL). We examine two issues of distributed representation in these TD algorithms: distributed representations of belief and distributed discounting factors. Distributed representation of belief allows the believed state of the world to distribute across sets of equivalent states. Distributed exponential discounting factors produce hyperbolic discounting in the behavior of the agent itself. We examine these issues in the context of a TD RL model in which state-belief is distributed over a set of exponentially-discounting “micro-Agents”, each of which has a separate discounting factor (γ). Each µAgent maintains an independent hypothesis about the state of the world, and a separate value-estimate of taking actions within that hypothesized state. The overall agent thus instantiates a flexible representation of an evolving world-state. As with other TD models, the value-error (δ) signal within the model matches dopamine signals recorded from animals in standard conditioning reward-paradigms. The distributed representation of belief provides an explanation for the decrease in dopamine at the conditioned stimulus seen in overtrained animals, for the differences between trace and delay conditioning, and for transient bursts of dopamine seen at movement initiation. Because each µAgent also includes its own exponential discounting factor, the overall agent shows hyperbolic discounting, consistent with behavioral experiments

Detailed Analysis of <em>ITPR1 </em>Missense Variants Guides Diagnostics and Therapeutic Design

\ua9 2023 The Authors. Movement Disorders published by Wiley Periodicals LLC on behalf of International Parkinson and Movement Disorder Society.Background: The ITPR1 gene encodes the inositol 1,4,5-trisphosphate (IP3) receptor type 1 (IP3R1), a critical player in cerebellar intracellular calcium signaling. Pathogenic missense variants in ITPR1 cause congenital spinocerebellar ataxia type 29 (SCA29), Gillespie syndrome (GLSP), and severe pontine/cerebellar hypoplasia. The pathophysiological basis of the different phenotypes is poorly understood. Objectives: We aimed to identify novel SCA29 and GLSP cases to define core phenotypes, describe the spectrum of missense variation across ITPR1, standardize the ITPR1 variant nomenclature, and investigate disease progression in relation to cerebellar atrophy. Methods: Cases were identified using next-generation sequencing through the Deciphering Developmental Disorders study, the 100,000 Genomes project, and clinical collaborations. ITPR1 alternative splicing in the human cerebellum was investigated by quantitative polymerase chain reaction. Results: We report the largest, multinational case series of 46 patients with 28 unique ITPR1 missense variants. Variants clustered in functional domains of the protein, especially in the N-terminal IP3-binding domain, the carbonic anhydrase 8 (CA8)-binding region, and the C-terminal transmembrane channel domain. Variants outside these domains were of questionable clinical significance. Standardized transcript annotation, based on our ITPR1 transcript expression data, greatly facilitated analysis. Genotype–phenotype associations were highly variable. Importantly, while cerebellar atrophy was common, cerebellar volume loss did not correlate with symptom progression. Conclusions: This dataset represents the largest cohort of patients with ITPR1 missense variants, expanding the clinical spectrum of SCA29 and GLSP. Standardized transcript annotation is essential for future reporting. Our findings will aid in diagnostic interpretation in the clinic and guide selection of variants for preclinical studies. \ua9 2023 The Authors. Movement Disorders published by Wiley Periodicals LLC on behalf of International Parkinson and Movement Disorder Society

Routine human papillomavirus genotyping by DNA sequencing in community hospital laboratories

<p>Abstract</p> <p>Background</p> <p>Human papillomavirus (HPV) genotyping is important for following up patients with persistent HPV infection and for evaluation of prevention strategy for the individual patients to be immunized with type-specific HPV vaccines. The aim of this study was to optimize a robust "low-temperature" (LoTemp™) PCR system to streamline the research protocols for HPV DNA nested PCR-amplification followed by genotyping with direct DNA sequencing. The protocol optimization facilitates transferring this molecular technology into clinical laboratory practice. In particular, lowering the temperature by 10°C at each step of thermocycling during <it>in vitro </it>DNA amplification yields more homogeneous PCR products. With this protocol, template purification before enzymatic cycle primer extensions is no longer necessary.</p> <p>Results</p> <p>The HPV genomic DNA extracted from liquid-based alcohol-preserved cervicovaginal cells was first amplified by the consensus MY09/MY11 primer pair followed by nested PCR with GP5+/GP6+ primers. The 150 bp nested PCR products were subjected to direct DNA sequencing. The hypervariable 34–50 bp DNA sequence downstream of the GP5+ primer site was compared to the known HPV DNA sequences stored in the GenBank using on-line BLAST for genotyping. The LoTemp™ ready-to-use PCR polymerase reagents proved to be stable at room temperature for at least 6 weeks. Nested PCR detected 107 isolates of HPV in 513 cervicovaginal clinical samples, all validated by DNA sequencing. HPV-16 was the most prevalent genotype constituting 29 of 107 positive cases (27.2%), followed by HPV-56 (8.5%). For comparison, Digene HC2 test detected 62.6% of the 107 HPV isolates and returned 11 (37.9%) of the 29 HPV-16 positive cases as "positive for high-risk HPV".</p> <p>Conclusion</p> <p>The LoTemp™ ready-to-use PCR polymerase system which allows thermocycling at 85°C for denaturing, 40°C for annealing and 65°C for primer extension can be adapted for target HPV DNA amplification by nested PCR and for preparation of clinical materials for genotyping by direct DNA sequencing. HPV genotyping is performed by on-line BLAST algorithm of a hypervariable L1 region. The DNA sequence is included in each report to the physician for comparison in following up patients with persistent HPV infection, a recognized tumor promoter in cancer induction.</p

STDP Allows Fast Rate-Modulated Coding with Poisson-Like Spike Trains

Spike timing-dependent plasticity (STDP) has been shown to enable single neurons to detect repeatedly presented spatiotemporal spike patterns. This holds even when such patterns are embedded in equally dense random spiking activity, that is, in the absence of external reference times such as a stimulus onset. Here we demonstrate, both analytically and numerically, that STDP can also learn repeating rate-modulated patterns, which have received more experimental evidence, for example, through post-stimulus time histograms (PSTHs). Each input spike train is generated from a rate function using a stochastic sampling mechanism, chosen to be an inhomogeneous Poisson process here. Learning is feasible provided significant covarying rate modulations occur within the typical timescale of STDP (∼10–20 ms) for sufficiently many inputs (∼100 among 1000 in our simulations), a condition that is met by many experimental PSTHs. Repeated pattern presentations induce spike-time correlations that are captured by STDP. Despite imprecise input spike times and even variable spike counts, a single trained neuron robustly detects the pattern just a few milliseconds after its presentation. Therefore, temporal imprecision and Poisson-like firing variability are not an obstacle to fast temporal coding. STDP provides an appealing mechanism to learn such rate patterns, which, beyond sensory processing, may also be involved in many cognitive tasks

Alendronate decreases orthotopic PC-3 prostate tumor growth and metastasis to prostate-draining lymph nodes in nude mice

<p>Abstract</p> <p>Background</p> <p>Metastatic prostate cancer is associated with a high morbidity and mortality but the spreading mechanisms are still poorly understood. The aminobisphosphonate alendronate, used to reduce bone loss, has also been shown to inhibit the invasion and migration of prostate cancer cells <it>in vitro</it>. We used a modified orthotopic PC-3 nude mouse tumor model of human prostate cancer to study whether alendronate affects prostate tumor growth and metastasis.</p> <p>Methods</p> <p>PC-3 cells (5 × 10⁵) were implanted in the prostates of nude mice and the mice were treated with alendronate (0.5 mg/kg/day in PBS, s.c.) or vehicle for 4 weeks. After sacrifice, the sizes of tumor-bearing prostates were measured and the tumors and prostate-draining regional iliac and sacral lymph nodes were excised for studies on markers of proliferation, apoptosis, angiogenesis and lymphangiogenesis, using histomorphometry and immunohistochemistry.</p> <p>Results</p> <p>Tumor occurrence in the prostate was 73% in the alendronate-treated group and 81% in the control group. Mean tumor size (218 mm³, range: 96–485 mm³, <it>n </it>= 11) in the alendronate-treated mice was 41% of that in the control mice (513 mm³, range: 209–1350 mm³, <it>n </it>= 13) (<it>p </it>< 0.05). In the iliac and sacral lymph nodes of alendronate-treated mice, the proportion of metastatic area was only about 10% of that in control mice (<it>p </it>< 0.001). Immunohistochemical staining of tumor sections showed that alendronate treatment caused a marked decrease in the number of CD34-positive endothelial cells in tumors (<it>p </it>< 0.001) and an increase in that of ISEL positive apoptotic cells in tumors as well as in lymph node metastases (<it>p </it>< 0.05) compared with those in the vehicle-treated mice. The density of m-LYVE-1-stained lymphatic capillaries was not changed.</p> <p>Conclusion</p> <p>Our results demonstrate that alendronate treatment opposes growth of orthotopic PC-3 tumors and decreases tumor metastasis to prostate-draining lymph nodes. This effect could be at least partly explained by decreased angiogenesis and increased apoptosis. The results suggest that bisphosphonates have anti-tumoral and anti-invasive effects on primary prostate cancer.</p